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Abstract 

Background: A randomized, controlled trial, intended to include 460 patients, is currently studying peroperative 
goal-directed hemodynamic treatment (GDHT) of aged hip-fracture patients. Interim efficacy analysis performed on 
the first 100 patients was statistically uncertain; thus, the trial is continuing in accordance with the trial protocol. 
This raised the present investigation's main question: Is it reasonable to continue to fund the trial to decrease 
uncertainty? To answer this question, a previously developed probabilistic cost-effectiveness model was used. That 
model depicts (1) a choice between routine fluid treatment and GDHT, given uncertainty of current evidence and 
(2) the monetary value of further data collection to decrease uncertainty. This monetary value, that is, the expected 
value of perfect information (EVPI), could be used to compare future research costs. Thus, the primary aim of the 
present investigation was to analyze EVPI of an ongoing trial with interim efficacy observed. 

Methods: A previously developed probabilistic decision analytic cost-effectiveness model was employed to 
compare the routine fluid treatment to GDHT. Results from the interim analysis, published trials, the meta-analysis, 
and the registry data were used as model inputs. EVPI was predicted using (1) combined uncertainty of model 
inputs; (2) threshold value of society's willingness to pay for one, quality-adjusted life-year; and (3) estimated 
number of future patients exposed to choice between GDHT and routine fluid treatment during the expected 
lifetime of GDHT. 

Results: If a decision to use GDHT were based on cost-effectiveness, then the decision would have a substantial 
degree of uncertainty. Assuming a 5-year lifetime of GDHT in clinical practice, the number of patients who would 
be subject to future decisions was 30,400. EVPI per patient would be €204 at a €20,000 threshold value of society's 
willingness to pay for one quality-adjusted life-year. Given a future population of 30,400 individuals, total EVPI 
would be €6.19 million. 

Conclusions: If future trial costs are below EVPI, further data collection is potentially cost-effective. When applying a 
cost-effectiveness model, statements such as 'further research is needed' are replaced with 'further research is cost- 
effective and 'further funding of a trial is justified'. 

Trial registration: ClinicalTrials.gov NCT01 141894 
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Background 

Before conducting costly clinical trials, researchers may 
need to estimate trial costs and convince funding bodies 
that returns on investment are acceptable. Large clinical 
trial protocols include an interim analysis plan, whose 
objectives usually cover (1) safety, efficacy, or ethical is- 
sues and (2) assessment of expected difficulties with pa- 
tient enrolment that could influence the trials costs and 
the credibility of its results. Interim analyses of efficacy 
may or may not be conclusive, depending on the 
detected effect size and the pre-trial sample size estima- 
tion. However, interim analysis results may influence 
investigators; consequently, standards for managing in- 
terim results and for implementing various statistical 
analysis methods have been recommended [1,2]. When 
the primary outcomes statistical uncertainty, as revealed 
by interim analysis, indicates need for further data 
collection, researchers may ask the question: Is it rea- 
sonable to continue to fund the trial to decrease uncer- 
tainty? Increasingly, resource allocation in health care 
considers the principles of cost-effectiveness in different 
jurisdictions; cost-effectiveness has become a key criterion 
for decision makers when deciding which health-care in- 
terventions should be made available in collectively 
funded health-care systems [3]. Because the purpose of a 
clinical trial is to reach a decision [4], the principles of 
cost-effectiveness and value for money could be, and per- 
haps should be, applied to decide, after the interim ana- 
lysis, if a clinical trial should be extended. The question 
then becomes whether current data from the interim ana- 
lysis are sufficient or whether data collection should con- 
tinue to decrease uncertainty regarding treatment choices. 
Further, when deciding whether or not to introduce a new 
technology, it is recommended that such a decision be 
made by considering the cost-effectiveness of the technol- 
ogy, taking into account all existing evidence and whether 
additional research is itself efficient. Thus, there is suffi- 
cient evidence regarding the cost-effectiveness of an inter- 
vention when the costs of undertaking additional research 
are greater than the benefits of reducing uncertainty [3]. 
Extending this rationale to clinical trials, after interim ana- 
lysis, a trial should be continued when the costs of under- 
taking additional research are less than its benefits, in 
terms of reducing uncertainty. This article describes 
such an interim analysis of a randomized controlled clin- 
ical trial of aged hip-fracture patients (ClinicalTrials.gov 
NCT01 141894) and the estimation of the value of 
extending the trial on the basis of value of information 
analysis. The trials objective was to compare costs and 
consequences of routine peroperative fluid treatment with 
goal-directed hemodynamic treatment (GDHT). The in- 
terim analysis during the trial determined that, due to stat- 
istical uncertainty, further data collection was required. 
Thus our present investigations primary and secondary 



aims were to (1) analyze expected value of perfect infor- 
mation (EVPI) when studying GDHT, and (2) provide 
decision support material when stakeholders must deter- 
mine if it is reasonable to fund further data collection after 
the interim analysis. 

Our investigation modeled cost-effectiveness and esti- 
mated result uncertainty of the clinical trial. Our model 
depicted (1) the choice between treatment options, given 
current, best available evidence; and (2) the monetary 
value of a decision to collect further information, via fu- 
ture clinical research, to decrease uncertainty. Here, 
monetary value is expressed as a concept called expected 
value of perfect information (EVPI) [5,6]. 

Methods 

This section describes (1) the clinical trials design and 
efficacy data extracted from the interim analysis, (2) the 
decision-analytic model and model inputs used in our 
analyses, and (3) the EVPI estimation method. 

Design of the clinical trial, interim analysis efficacy data 

Our data source was a single-centre, open, randomized 
(1:1), and controlled, parallel-group, superiority clinical 
trial. The trial is on-going at a primary teaching hospital, 
Karolinska University Hospital, Huddinge, Sweden. Eli- 
gible patients (aged > 70 years, weight > 40 kg) are those 
scheduled for hip-fracture surgery during regular operat- 
ing hours. Patients were enrolled if a signed informed 
consent was obtained after they received information 
about trial-specific procedures. The local research ethics 
committee approved the trial (ID: 2008-1240-31), and 
Sweden's Medical Products Agency authorised the trial 
(MPA ID; 151:2009/81083). 

Trial objectives 

Primary: to identify the absolute and relative risk of 
post-operative complications at hospital discharge. Sec- 
ondary: to analyze cost-effectiveness of GDHT compared 
with routine fluid treatment. 

Statistics 

Using expected absolute risk (0.61) [7] and relative risk 
(0.63) [8] for post-operative morbidity (GDHT compared 
with routine fluid treatment), a sample size calculation 
was made (80% power, the type I error <0.05). Assuming 
that 57% of complications could be reduced due to pa- 
tients' age and comorbidity [9], a 460-patient sample size 
is required. For the analysis Statistical software was used 
(version 10. StatSoft Inc, Tulsa, OK, USA). 

Interventions 

Interventions included the following: (1) GDHT to attain 
oxygen delivery index >600 mL/ min" 1 m" 2 (using fluids 
and dobutamine) and (2) a protocol-guided routine fluid 
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treatment. Both groups were monitored with the 
LiDCO™ monitor ( LiDCO Ltd., Sawston, Cambridge, 
United Kingdom). 

The effect size per interim analysis 

This article reports relative risk of post-operative com- 
plications from the trial because these data were speci- 
fied in the interim analysis strategy. See Figure 1 for the 
inclusion sequence and Table 1 for patient characteris- 
tics. The post-operative outcome is expressed by relative 
risk (95% confidence intervals) taken from an intention- 
to-treat analysis. In-hospital mortality (n = 3) contrib- 
uted to post-operative complications. 

The model and model inputs used in this investigation 

A decision-analytic probabilistic cost-effectiveness model 
(Figure 2) was developed. The model illustrates the con- 
sequences of use of GDHT or routine fluid treatment 
for a hypothetical cohort of patients after hip fracture dur- 
ing 5 post-operative years [10]. Model inputs consisted of 
these estimates: (1) probability of post-operative morbidity 
and mortality, (2) health- related quality of life, and (3) 
costs. Model outputs consisted of (1) aggregated health- 
care costs and (2) quality-adjusted life-years (QALYs). The 
model was programmed and analyzed using Microsoft 
Excel (2007, version 12.0.6425.1000). The analysis applied 
Swedish hospital costs that were converted to euros using 
the exchange rate of €1 = SEK 9.41. Costs and QALYs 
were discounted by 3% annually. 

More comprehensive descriptions of the model struc- 
ture and data used for model parameters are available 
elsewhere [10] (Additional files 1 and 2). The model 
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Figure 1 Inclusion sequence of the first 100 randomized 

patients in the trial: ClinicalTrials.gov NCT01 141894. 

\ ) 



contained two parts: a decision tree, used to estimate 
short-term costs and effects, and a Markov structure, 
used to estimate long-term costs and effectsThe 
decision-tree in Figure 2A illustrates short-term out- 
come for hypothetical patients. The tree starts with the 
decision (rectangle) between the two alternatives, 
followed by the chance node (circle), whereby the 
patients can transition (by chance) along the arrows to 
selected, post-operative outcomes (triangles). These out- 
comes are as follows: uncomplicated recovery; cardiovas- 
cular complications; stroke; other complications (that is, 
pulmonary or urinary tract infections, post-operative 
confusion, kidney insufficiency, wound infections, and 
pulmonary embolism); or death. 

Model inputs in the decision tree 

The model inputs in the decision tree (Table 2) 
consisted of (1) probability estimates of the post- 
operative outcome, (2) estimates of the post-operative 
health state (expressed by QALY weights), and (3) costs. 

Probability estimates for routine fluid treatment, by 
type of post-operative complication, could not be calcu- 
lated from the interim data due to the low number of 
observations per complication (n <5). Instead, these esti- 
mates were extracted from a Swedish patient cohort 
with hip fracture. [7] For GDHT, the estimate of relative 
risk from the interim analysis was applied to the prob- 
ability estimates for the routine fluid treatment by type 
of post-operative outcome (Table 2). Mortality estimates 
could not be calculated from the interim data due to the 
low number of observations (n <5); instead, published 
meta-analysis data were used [8]. 

Pre-fracture QALY weights were obtained from an 
age-matched general population [11] (aged >80 years). 
Post-fracture QALY weights associated to the selected 
post-operative outcomes were unavailable, so QALY 
weights were obtained from a representative sample with 
the disease in the population [11,12] and from a longitu- 
dinal Swedish clinical trial [13]. 

The long-term model (Markov structure) extrapolated 
the effects of the post-operative outcome to post- 
operative health conditions and the influence of the out- 
come on long-term survival (Figure 2B). Hypothetical 
patients transitioned among these health conditions or 
stayed in the current condition during the cycles in 1 
year (thatis, a Markov cycle). Recovery is allowed from 
the 'other complications' state; we assumed that patients 
continued to live with cardiovascular complications or 
stroke after the initial event. During each Markov cycle, 
quality of life and number of survivors decrease. 

Model inputs in the Markov structure 

Model inputs in the long-term model (Table 2) consisted 
of survival data, QALY weights, and cost data. 
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Table 1 Patient characteristics and interim efficacy data; values are absolute or mean ± SD 

Patient characteristics GDHT 



Routine fluid treatment 



Number allocated 49 

Age, years (mean) 86 (± 7) 

Sex, male/female 13/36 

American Society of Anesthesiologists' grading (1/2/3/4) 0/13/31/5 

Number of patients with complications (including in-hospital mortality) 15 
Relative risk of GDHT compared to routine fluid treatment based on intention to treat (95% CI) 



50 
85 ( ± 7) 

9/41 
1/15/29/5 

19 



0.806 (0.464 to 1 .397) 



CI confidence interval, GDHT goal-directed hemodynamic therapy. 

For patients without complication, age-adjusted stand- 
ard mortality was used. For patients with complications, 
disease-related mortality was used (non-fractured patients 
with cardiovascular, cerebrovascular or with other diseases). 
In instances of cardiovascular complications or stroke, 
survival was estimated using age- and disease-related 
mortality from the Swedish National Stroke Registry 
[14] and the Swedish National Registry on Secondary 
Prevention in Cardiac Intensive Care (Kalle Spangberg, 
Ph.D., section manager, Uppsala Clinical Research Center, 
Uppsala University, Uppsala, Sweden, written communi- 
cation: 15 May 2009). 

QALY weights associated with short-term clinical out- 
comes were allocated in the early simulation in the Markov 
structure. The post-fracture QALY weights were multiplied 
by the time spent in the current state of health, which 
resulted in the number of QALYs. 



Mean costs of inpatient and outpatient long-term med- 
ical care services were obtained from the Epidemiological 
Centre of the Swedish National Board of Health and 
Welfare for patients undergoing surgery for hip fracture 
in 2007 and hospitalized during 2008 (Leif Forsberg, 
statistician, Department of Statistics, Monitoring and 
Evaluation, Swedish National Board of Health, Stockholm, 
Sweden, written communication: 7 December 2009). 
Hospital costs for each post-operative complication were 
acquired from individual patient-specific cost data obtained 
from University Hospital in Lund, Sweden. 

Assumptions in the model 

The following assumptions were made in the model: 

1. Hypothetical patients could have one post-operative 
complication. 




Figure 2 Cost-effectiveness model. A) Short-term model, the decision tree. The arrows represent the transition of the hypothetical patients 
towards the selected post-operative outcomes (triangles). These transitions are characterized by probability estimates (p1 to p10), costs, and 
health-related quality-of-life weights. For the routine fluid treatment, probability estimates were extracted from a cohort from Lund University 
Hospital [7]. * For goal-directed hemodynamic treatment (GDHT), the interim analysis was used. ** For mortality, published data on high-risk 
patients were used [8]. B) The long-term model, Markov structure. The hypothetical patients were allocated to health states characterized by 
health-related quality-of-life weights. During annual cycles of simulation, the patients transition in the model or stay in the same heath state. 
These transitions are characterized by probability estimates (pi 1 to p21). {AU After each cycle, quality-adjusted life-years and direct health-care 
costs are aggregated. 
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Table 2 Model inputs 



Table 2 Model inputs (Continued) 



Model inputs 



Estimates Distributions 



Short-term clinical outcome (routine care) 3 
Probability of... 

In-hospital mortality 
Cardiovascular complications 
Stroke 

Other complications 

uncomplicated recovery 

Relative risk of mortality GDHT versus 
routine [8] 

Relative risk of complications GDHT 
versus routine 

Long-term clinical outcome 
(routine care) 

Mortality associated with cardiovascular 
disease 6 

First year 

Second year 

Third year 

Mortality associated with stroke at 3 
months c 

Mortality associated with other 
complications 3 

Recovery associated with other 
complications 3 

Mortality after recovery with other 
complications 3 

Costs/patient for routine fluid treatment 
in the clinical routine (€) d 

Medical device for fluid treatment 

Human resources in pre-operative 
area 

Human resources during anesthesia 

Costs/patient for GDHT (€) d 

Medical device for GDHT 

Human resources in pre-operative area 

Human resources during anesthesia 

Post-operative direct health-care costs/ 
patient (€) e 

Cardiovascular complications 

Myocardial infarction 

Heart failure 
Stroke 

Other complications 
Pneumonia 
Renal failure 
Wound infection 
Deep-vein thrombosis 
Pulmonary embolism 



0.129 Dirichlet 1 (52, 26, 
0.065 2 ' 162 ' 160) 
0.005 
0.403 
0.398 
0.75 



0.81 



0.107 
0.058 
0.056 
0.15 

0.18 

0.41 

0.15 



11 

27 

117 

221 
159 
401 



Lognormal 
(-0.28; 0.09) 

Lognormal 



Deterministic 
Deterministic 
Deterministic 
Deterministic 

Beta (31; 140) 

Beta (70; 101) 

Beta (17; 95) 



Deterministic 
Deterministic 

Deterministic 

Deterministic 
Deterministic 
Deterministic 



7,498 Gamma (90; 83) 

9,903 Gamma (104;95) 

7,550 Gamma (8; 956) 

8,514 Gamma (106; 81) 

12,197 Gamma (6; 1442) 

8,566 Gamma (218; 39) 

7,617 Gamma (62; 124) 

10,190 Gamma (17, 600) 



Gastrointestinal bleeding 
Confusion 
Death 

No complications 

Direct health-care costs, first year after 
hospital stay (€) f 

State after. . . 

No complications 

Cardiovascular complications 

Stroke 

Other complications 
Recovery from other complications 
Death 

Direct health-care costs, 2 to 10 year 
after hospital stay (€) f 

State after... 

Cardiovascular complications 

Stroke 

Other complications 

QALY weights, estimates [7] 

>80 years age 

Recovered after other 
complication [9] 

Decrements of QALY weights [8] 
State after... 

Cardiovascular complications 

Stroke 

Other complications 



9,900 Gamma (64, 154) 

7,961 Gamma (866; 9) 

9,020 Gamma (273, 33) 

6,753 Gamma (956; 7) 



147 
7,673 
7,512 
7,314 

396 
4,837 



386 
402 
396 



Deterministic 
Deterministic 
Deterministic 
Deterministic 
Deterministic 
Deterministic 



Deterministic 
Deterministic 
Deterministic 



0.74 Beta (322; 113) 
0.66 Beta (227; 1 1 7) 



-0.19 
-0.35 
-0.15 



Gamma (298; 
0.0006) 

Gamma (100; 
0.0035) 

Gamma (100; 
0.0007) 



GDHT goal-directed hemodynamic therapy, QALY quality-adjusted life years, 
€ Euros. 

1 The Dirichlet distribution is a multivariate normalization of beta distribution 
that considers that the sum of probabilities is 1.0. 
a Swedish Registry on Hip Fracture. 

b Swedish National Registry on Secondary Prevention in Cardiac 
Intensive Care. 

c Swedish National Stroke Registry. 

d Karolinska University Hospital in Huddinge, Sweden. 

e University Hospital in Lund, Sweden. 

f Epidemiological Centre of the Swedish National Board of Health. 
Probability estimates characterized each post-operative outcome 
(the post-operative complications as well as death). 

2. Patients experiencing a complication had a decline 
of quality of life during the 5 post-operative years 
equal to that of patients in the non-fractured 
population who had cardiovascular complications or 
stroke. 

3. The 5-year survival rate after uncomplicated 
recovery for patients with hip fracture was the 
same as that for aged non-fractured patients 
(standard mortality). 
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4. The 5 -year survival rate for post-operative 

cardiovascular complications and stroke in patients 
with hip fracture was the same as that in non- 
fractured patients with cardiovascular complications. 

Probabilistic sensitivity analysis 

To incorporate the uncertainty of model parameters in 
the probabilistic sensitivity analysis, each parameter as- 
sociated with statistical uncertainty was defined with 
probability distributions. The choice of distributions was 
guided by recent recommendations by Briggs et al [6] 
and was parameterized with the mean and standard 
error. The values of QALY weights are constrained on 
the interval 0 to 1 and for these beta distributions were 
used. The transition probabilities are mutually exclusive 
events (multinomial data) and for these Dirichlet distri- 
bution was used that is a multivariate generalization of 
beta distribution. The skewed cost data and the decre- 
ments of QALY weights constrained on the interval 0 to 
positive infinity, and for these parameters gamma distribu- 
tion was used. The relative risk (ratio) was transformed 
into a log form and log-normal distributions were used. In 
Table 2 the distributions used are listed for each param- 
eter. The long-term costs were based on fixed price lists, 
and the survival rates after cardiovascular complications 
or stroke were extracted from registries on large popula- 
tions with low standard error (Table 2). Thus, both these 
sets of parameters were incorporated as deterministic 
probabilities in the model. 

To propagate the uncertainty of the parameters 
through to the cost-effectiveness estimate, a second- 
order Monte Carlo simulation was used; the cohort was 
simulated through 5 Markov cycles (years). The input 
parameters were sampled using the distributions given 
in Table 2. In each simulation, the set of model input 



values was randomly drawn from the model parameter s 
defined probability distributions. The simulation then 
was performed 1,000 times [6], generating 1,000 estimates 
of aggregated costs and QALYs. In Figure 3, incremental 
costs (AC = Cost G DHT - Cost Rout ine) are plotted against 
incremental effects (AE = Effect GDH T - Effect Routine ). 
For 91.5% of simulations, GDHT (compared with routine 
fluid treatment) was better and less costly (AE positive 
and AC negative) or was better and more costly (both AE 
and AC positive). 

Expected value of perfect information estimation method 

As shown in Figure 3, the model propagated parameter 
uncertainty onto the uncertainty of cost-effectiveness. If 
a decision was to be made regarding adoption of 
GDHTon the basis of expected costs and QALYs from 
the current evidence, this decision would be associated 
with uncertainty and there would be a risk of making an 
incorrect decision. In Figure 3, the number of values in 
upper-left quadrant (GDHT more costly and less effect- 
ive) contributes to the probability of making such a 
decsion. In addition, some values in the upper-right 
quadrant also may represent an incorrect decision - 
despite GDHT resulting in positive incremental effect 
(AE) - because the cost per QALY may be perceived as 
being too high. The primary decision-making factor in 
this quadrant is a threshold value of how much society 
is willing to pay for 1 additional life-year with full health 
in the target population. The slope of the dotted line (X) 
on Figure 3 illustrates a value of €20,000 per QALY. The 
line represents one possible threshold value (or cost- 
effectiveness threshold); values above the dotted line 
represent an incorrect decision. 

The combination of uncertainty of model inputs and 
society's willingness to pay for 1 additional life-year 
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Figure 3 Incremental costs and effects {LQALY) of goal-directed hemodynamic treatment (GDHT) versus routine fluid therapy. The 

dotted line represents one threshold value of how much society would be willing to pay for 1 additional life-year with full health for each patient 
in the target population. 
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with full health influences the decision. The probabilities 
of making an incorrect decision are quantified, given the 
uncertainty of model inputs and the consequences of the 
relinquished or foregone benefit if GDHT were adopted. 

The purpose of further data collection is to eliminate 
risk of relinquished benefit. In practice, the value of 
this relinquished benefit is the EVPI, whereby a deci- 
sion is made for one future patient. The EVPI must be 
calculated for all potential future clinical decisions on 
all future patients who will gain by the advantage of 
additional research during the treatments (for example, 
GDHT) expected lifetime in clinical use. Therefore, the 
effective population, that is, the population that may 
benefit in the future during the treatment s lifetime, is 
estimated (and the discount rate is applied). The total 
EVPI population is calculated by multiplying the EVPI 
per patient by the population that might benefit in the 
future (effective population): 

Population EVPI = EVPI patient x effective population 



Results 

Figure 3 illustrates the model outputs: expected costs 
and effects. It shows that in 76.2% of the simulations, 
the GDHT is less costly and more effective (negative 
incremental cost, positive incremental effect); in 17.1% 
of simulations, GDHT is more costly and more effective 
(positive incremental cost and positive incremental effect); 
and in 6.7% of simulations, GDHT more expensive and 
less effective (positive incremental cost and negative 
incremental effect). 



Expected value of perfect information 

At a cost-effectiveness threshold (X) of €50,000, the EVPI 
is €337 per patient. At a threshold of €20,000, the EVPI 
per patient is €204. Given 6,440 operations annually in 
Sweden (for patients aged >80 years), a conservative 
expected lifetime of 5 years for the GDHT technology, 
and a 3% discount per year, the EVPI for the effective 
population (30,378 individuals) is €6.19 million to €10.2 
million when the cost-effectiveness threshold is between 
€20,000 and €50,000. (The authors assumed that new 
technologies or other innovations may influence clinical 
practice and that GDHT may not be the current trend 
after 5 years.) The model was run using varying values 
of a cost-effectiveness threshold, which generated vary- 
ing expected values of perfect information (Figure 4). 

Discussion 

After classical statistical inference analysis of the interim 
efficacy data, our model showed that the GDHT should 
not be adopted in the clinical routine because the null 
hypothesis (relative risk = 1.0) was within limits of the 
95% confidence interval (Table 1). However, the cost- 
effectiveness analysis and its accompanying probabilistic 
sensitivity analysis, taking existing evidence into account, 
showed that, in 93.3% of the simulations, treating pa- 
tients with GDHT, when compared to routine fluid 
treatment, resulted in QALYs gained (Figure 3). How- 
ever, if a decision was made on the basis of current in- 
formation, some iteration found that the decision could 
be incorrect where there are higher health-care costs 
and less effect (upper-left quadrant on Figure 3) or when 
costs are above the cost-effectiveness threshold (upper- 
right quadrant). The EVPI is the value of the incorrect 
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Figure 4 The expected value of further information for the Swedish patient population aged >80 years with hip fracture. The expected 
value of perfect information (EVPI) is plotted against the willingness to pay per quality-adjusted life-year (cost-effectiveness threshold). 
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decision as it is the maximum acceptable value that soci- 
ety should pay to eliminate decision uncertainty via fur- 
ther data collection. The cost-effectiveness is a related 
value that society is willing to pay for 1 additional life- 
year with full health (cost-effectiveness threshold, X) for 
each patient in the population of interest. In Sweden, 
there is no fixed officially accepted threshold or range, 
but a cost-effectiveness threshold of between €20,000 
and €50,000 has been discussed in this paper; in excep- 
tional cases, this threshold can be even higher. Severity 
of the health condition under consideration is the major 
reason for accepting varying thresholds [15-20]. With a 
broad range of alternative implicit thresholds for society's 
willingness to pay for one QALY, we chose a conservative 
interpretation in drawing our conclusions. For the broad 
range of cost effectiveness thresholds used in this analysis, 
we found that the EVPI is in the interval of €6.19 to 
€10.2 million. 

The rationale of probabilistic modeling 

The rationale of probabilistic modeling is to reflect the 
uncertainty of the model inputs and to describe the un- 
certainty about the model outputs. In non-linear models, 
the model outputs are the result of multiplicative or 
power functions. The interest of a decision-maker is the 
expected distribution of model outputs. This cannot be 
obtained by analyzing the upper-limit and lower-limit 
95% confidence intervals of the model inputs [21]. In- 
stead, the uncertainty of model outputs is obtained by 
probabilistic sensitivity analyses [21]. The uncertainty of 
model outputs implies the possibility of an incorrect deci- 
sion that results in costs (forgone benefits). In the decision 
theoretic approach, a value is ascribed to the reduction of 
uncertainty (creation of additional information) and a deci- 
sion may include the option to acquire additional informa- 
tion. It is important to note that the current calculated 
EVPI is conditional on model uncertainty due to the 
assumptions made (and listed in the methods) about the 
model structure and the data used. Such assumptions are 
not accounted for in the distributions of the parameters 
because they originate from active choices made rather 
than from statistical uncertainty. The effect of our four 
assumptions on mean cost-effectiveness and its uncertainty 
was investigated in the previously published model [10] 
and was found to have a marginal influence on the results. 
Thus, our assumptions have not been investigated here 
with regard to their influence on EVPI. 

The rationale of the expected value of perfect 
information 

EVPI implies that data are obtained from an infinite 
sample that removes all uncertainty and places an upper 
limit of the value of further research, given that add- 
itional data may reduce uncertainty. In reality, the value 



of further data collection resulting in new information is 
most likely far less. Further research may yield partial in- 
formation on the parameters of interest (not an infinite 
sample). The value of new information thus depends on 
the uncertainty of the new parameter estimates. With 
perfect information, a decision-maker would know how 
the uncertainties would resolve and could make the cor- 
rect decision. The better decision would be to choose a 
treatment with better effect at costs below a threshold 
value of what the society is willing to pay per gained 
QALY. EVPI can help decision makers identify which re- 
search is potentially cost-effective and which is not. It 
does not imply that research can be continued until the 
costs of research are below the EVPI. 

Publications regarding the EVPI method for interim 
analysis appear to be non-existent. In a broader perspec- 
tive, however, EVPI was recently used for two pilot stud- 
ies of research priorities in the United Kingdom [22-25]. 
In our study, we conclude that trial NCT01 141894 
should be continued, given the uncertain interim efficacy 
data and the rules of classical statistical inference. How- 
ever, recruitment time in this on-going trial has been 
prolonged for logistical reasons (almost 30% of the 
hip-facture surgeries initially planned during regular 
operating room hours were performed during off hours 
(65 out of 187 assessed for eligibility; see Figure 1). 
This factor increases trial cost; therefore, it is reason- 
able to address the monetary value of further research 
because it is not obvious that the trial should be fi- 
nanced for further data collection, given scarce clinical 
research resources. 

Interim analysis risks 

Sharing results of interim analyses could influence an 
on-going trial, and guidelines request that results from 
an interim analysis not be made public [1,2,26]. This is 
important because further recruitment could be affected 
and because patients could prefer any of the treatments, 
as seen in the preliminary analysis of efficacy and ad- 
verse events (although the intervention is not yet scien- 
tifically proven). In addition, the risk of introducing bias 
into the on-going trial could be substantial [26]. In line 
with the current recommendations, an interim analysis 
and the subsequent decision on further data collection 
should be performed by confidential data safety and 
monitoring committees. While an EVPI analysis could 
support such a decision or be used as an argument when 
applying for further funds, guidelines nevertheless sug- 
gest that the results of an interim analysis in such in- 
stances not be made public. However, the adherence to 
these guidelines has been questioned in the past, and the 
guidelines have been characterized as paternalistic [27]. 
Researchers in favor of making interim analyses public 
believe that it is important to inform society, patients, 
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and other researchers and to guide the design of future 
trials in considering safety issues and sample size. How- 
ever, it is important to note that current guidelines and 
statistical methods insist on a blind interim analysis 
[2,28]. The current trial was randomized but open and 
blinded during analysis. 

Conclusions 

After analysis of interim efficacy data, further data collec- 
tion in trial NCTO 1141894 is needed for statistical inference 
analysis. A further 4 years of recruitment is planned, with a 
12-month follow-up period. Even if predicted trial costs 
increase, it seems reasonable to fund the trial further, 
provided these costs remain below €6.19 million. 

Decisions on funding clinical trials usually do not utilize 
the results of health economic evaluations of the trials' in- 
terim analyses, even if the trials are very resource intensive 
and are funded by public resources. With use of the 
analytic framework, such as cost-effectiveness models, 
we can replace the statement 'further research is needed' 
with 'further research is cost- effective' and 'further funding 
of a trial is justified/ 

Additional files 
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